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PATENT APPLICATION 


SOUN DJYSII^ANDj^^ CAPTURI NG 
^NQ^PSODT ICINGJSOT TNDS OR TGTNATING 
FROM A PLURALITY OF SOUND SOURCES 


FIELD OF THE INVENTION 

The present invention generally relates to acoustical reproduction 
and sound field reconstruction. More specifically, it relates to methods and 
apparatus for separately recording a plurality of sounds produced concurrently by 
a plurality of sound sources and/or simultaneously reproducing a plurality of 
sounds separately recorded or produced by a plurality of sound sources. The 
invention also relates to methods and apparatus for sound production including 
controlling the interaction between a plurality of sounds based on relationships 
therebetween. 

BACKGROUND OF THE INVENTION 

Systems for recording and reproducing sounds produced by a 
plurality of sound sources are generally known. In the musical context, for 
example, systems for recording and reproducing live performances of bands and 
orchestras are known. In those cases, the sound sources are the musical 
instruments and performers' voices. More generally, however, a sound source is 
any object that produces sound. In a basic sense, sound is a series of physical 
disturbances in a medium (e.g., air). Typically, sound is created when an object 
(a sound source) vibrates, sending out a series of waves that propagate through air 
(or other media). In air, sound waves comprise fluctuations in air pressure above 
and below the normal atmospheric pressure (e.g., 14.7 psi). These fluctuations are 
refeiTed to as compressions and rarefactions. When compressions and rarefactions 
impinge upon our eardrums, we perceive sound. The greater the change in air 
pressure above and below normal atmospheric pressure, the greater the amplitude 
of the sound. Since most objects vibrate with a periodic back-and-forth motion or 
oscillation, most sound waves (and nearly all musical sounds) have a periodic 
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repetition, replicating the object's motion. Thus, a sound wave can be 
characterized by frequency and amplitude and can be represented generally by a 
sine wave. However, real sounds and musical signals are actually complex waves 
made up of many sound waves of different frequencies superimposed on one 
another. One reason for this is that a vibrating object (and therefore a sound wave 
produced by that object) includes a fundamental frequency (its lowest frequency) 
and overtones or harmonics which are a multiple of the fundamental frequency. 
The presence of these harmonics contribute to a musical instrument's 
characteristics, such as its timbre or tonal color. Thus, two instruments (e.g., a 
piano and a violin) both played at the same fundamental frequency will sound 
different because they have different harmonic structures. For example, a violin 
produces stronger harmonics that extend higher in frequency than that of the 
piano. 

Another factor that affects the perception of sound is phase. The 
term phase refers to the time relationship between two or more sound waves. A 
phase shift refers to a time displacement of a wave (e.g., a sine wave) relative to 
a fixed point. Phase shift has important consequences when sine waves are 
combined or superimposed. If two sine waves of equal frequency and the same 
phase are superimposed, their combination will create a wave of greater amplitude. 
If, however, one of the waves is phase-shifted by 180 degrees, then the two waves 
will cancel each other and produce no signal. 

Recording and reproducing sound produced by a sound source 
typically involves detecting sound waves produced by the sound source, 
converting the sound waves to audio signals (digital or analog), storing the audio 
signals on a recording medium and subsequently reading and amplifying the stored 
audio signals and supplying them as an input to one or more loudspeakers to 
reconvert the audio signals back to sound. Audio signals are typically electrical 
signals that correspond to actual sound waves, however this correspondence is 
"representative", not "congruent", due to various limitations intrinsic to the 
process of capturing and converting acoustical data. Other forms of audio signals 
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(e.g., optical), although more reliable in the transmission of acoustical data, 
encounter similar limitations due to capturing and converting the acoustical data 
from the original sound field. 

The reproduction of sound by use of loudspeakers typically involves 
moving a loudspeaker cone back and forth to recreate a pattern of compressions 
and rarefactions. The movement of the cone is controlled by inputting audio 
signals to a driver that drives the loudspeaker. As a result, the quality of the sound 
produced by a loudspeaker partly depends on the quality of the audio signal input 
to the loudspeaker, and partly depends on the ability of the loudspeaker to respond 
to the signal accurately. Ideally, to enable precise reproduction of sound, the 
audio signals should correspond exactly to (i.e., be a perfect representation of) the 
original sound and the reconversion of the audio signals back to sound should be 
a perfect conversion of the audio signal to sound waves. In practice however, such 
perfection has not been achieved due to various phenomenon that occur in the 
various stages of the recording/reproducing process, as well as deficiencies that 
exist in the design concept of "universal" loudspeakers. 

Additional problems are presented when trying to precisely record 
and reproduce sound produced by a plurality of sound sources. One significant 
problem encountered when trying to reproduce sounds from a plurality of sound 
sources is the inability of the system to recreate what is referred to as sound 
staging. Sound staging is the phenomena that enables a listener to perceive the 
apparent physical size and location of a musical presentation. The sound stage 
includes the physical properties of depth and width. These properties contribute 
to the ability to listen to an orchestra, for example, and be able to discern the 
relative position of different sound sources (e.g., instruments). However, many 
recording systems fail to precisely capture the sound staging effect when recording 
a plurality of sound sources. One reason for this is the methodology used by many 
systems. For example, such systems typically use one or more microphones to 
receive sound waves produced by a plurality of sound sources (e.g., drums, guitar, 
vocals, etc.) and convert the sound waves to electrical audio signals. When one 
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microphone is used, the sound waves from each of the sound sources are typically 
mixed (i.e., superimposed on one another) to form a composite signal. When a 
plurality of microphones are used, the plurality of audio signals are typically 
mixed (i.e., superimposed on one another) to form a composite signal. In either 
case the composite signal is then stored on a storage medium. The composite 
signal can be subsequently read from the storage medium and reproduced in an 
attempt to recreate the original sounds produced by the sound sources. However, 
the mixing of signals, among other things, limits the ability to recreate the sound 
staging of the plurality of sound sources. Thus, when signals are mixed, the 
reproduced sound fails to precisely recreate the original sounds. This is one 
reason why an orchestra sounds different when listened to live as compared with 
a recording. This is one major drawback of prior sound systems. Other problems 
are caused by mixing as well. 

While attempts have been made to address these drawbacks, none 
has adequately overcome the problem. For example, in some cases, the 
composite signal includes two separate channels (e.g., left and right) in an attempt 
to spatially separate the composite signal. In some cases, a third (e.g., center) or 
more channels (e.g., front and back) are used to achieve greater spatial separation 
of the original sounds produced by the plurality of sound sources. Two popular 
methodologies used to achieve a degree of spatial separation, especially in home 
theater audio systems, are Dolby Surround and Dolby Pro Logic. Dolby Pro Logic 
is the more sophisticated of the two and combines four audio channels into two for 
storage and then separates those two channels into four for playback over five 
loudspeakers. Specifically, a Dolby Pro Logic system starts with left, center and 
right channels across the front of the viewing area and a single surround channel 
at the rear. These four channels are stored as two channels, reconverted to four 
and played back over left, center and right front loudspeakers and a pair of 
monaural rear surround loudspeakers that are fed from a single audio channel. 
While this technique provides some measure of spatial separation, it fails to 
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precisely recreate the sound staging and suffers from other problems, including 
those identified above. 

Other techniques for creating spatial separation have been tried using 
a plurality of channels. However, regardless of the number of channels, such 
systems typically involve mixing audio signals to form one or more composite 
signals. Even systems touted as "discrete multi-channel", base the discreteness of 
each channel on a "directional component" (i.e., Dolby's AC-3, discrete 5. 1 multi- 
channel surround sound is based on five discrete directional channels and one 
omni-directional bass channel). "Directional components" help create a more 
engulfing acoustical effect, but do not address the critical losses of veracity within 
the audio signal itself. 

Other separation techniques are commonly used in an attempt to 
enhance the recreation of sound. For example, each loudspeaker typically 
includes a plurality of loudspeaker components, with each component dedicated 
to a particular frequency band to achieve a frequency distribution of the 
reproduced sounds. Commonly, such loudspeaker components include woofer or 
bass (lower frequencies), mid-range (moderate frequencies) and tweeters (higher 
frequencies). Components directed to other specific frequency bands are also 
known and may be used. When frequency distributed components are used for 
each of multiple channels (e.g., left and right), the output signal can exhibit a 
degree of both spatial distribution and frequency distribution in an attempt to 
reproduce the sounds produced by the plurality of sound sources. However, 
maximum recreation of the original sounds is not fully achieved. 

Another problem resulting from the mixing of either sounds 
produced by sound sources or the corresponding audio signals is that this mixing 
typically requires that these composite sounds or composite audio signals be 
played back over the same loudspeaker(s). It is well known that effects such as 
masking preclude the precise recreation of the original sounds. For example, 
masking can render one sound inaudible when accompanied by a louder sound. 
For example, the inability to hear a conversation in the presence of loud amplified 
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music is an example of masking. Masking is particularly problematic when the 
masking sound has a similar frequency to the masked sound. Other types of 
masking include loudspeaker masking, which occurs when a loudspeaker cone is 
driven by a composite signal as opposed to an audio signal corresponding to a 
single sound source. Thus, in the later case, the loudspeaker cone directs all of its 
energy to reproducing one isolated sound, as opposed to, in the former, the 
loudspeaker cone must "time-share" its energy to reproduce a composite of 
sounds simultaneously. 

Another problem with mixing sounds or audio signals and then 
amplifying the composite signal is intermodulation distortion. Intermodulation 
distortion refers to the fact that when a signal of two (or more) frequencies is input 
to an amplifier, the amplifier will output the two frequencies plus the sum and 
difference of these frequencies. Thus, if an amplifier input is a signal with a 
400Hz component and a 20KHz component, the output will be 400 Hz and 20 
KHz plus 19.6KHz (20KHz - 400Hz) and 20.4 KHz (20KHz + 400Hz). 

Another problem with existing loudspeakers is that they usually 
perform well at certain frequencies but not at others. Some are suited well for one 
type of music (e.g., rock), but not for others (e.g., a symphony). Furthermore, 
different frequency ranges require different levels of amplification to achieve an 
otherwise harmonious magnification. Current technology provides methods for 
suppressing such incongruences, but the methods are artificial and present a very 
limited linear solution to a nonlinear problem. Also, their directional qualities are 
limited. 

Thus, despite significant research and development, prior systems 
suffer various drawbacks and fail to maximize the ability of the system to 
precisely reproduce the original sounds. 
OBJECTS OF THE INVENTION 

It is an object of the present invention to overcome these and other 
drawbacks of the prior art. 
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It is another object of the present invention to provide an improved 
method and apparatus for recording and/or reproducing sounds produced by a 
plurality of sound sources. 

It is another object of the present invention to provide a method and 
apparatus for separately recording a plurality of sounds produced concurrently by 
a plurality of sound sources. 

It is another object of the present invention to provide a method and 
apparatus for simultaneously reproducing a plurality of separately recorded sounds 
or sounds produced by a plurality of sound sources. 

It is another object of the present invention to provide an improved 
recording and playback system capable of producing and reproducing sounds to 
attempt to recreate actual sounds produced by sound sources, and controlling the 
reproduction to take into account power variations of the various signals. 

It is another object of the present invention to provide an improved 
recording and playback system capable of capturing and reproducing sounds to 
recreate actual sounds produced by sound sources, where sounds from each of a 
plurality of sound sources (or a predetermined group of sources) are captured by 
separate sound detectors, and where the separately captured sounds are converted 
to audio signals, recorded, and played back by separately retrieving the stored 
audio signals from the recording medium and transmitting the retrieved audio 
signals separately to a separate loudspeaker system for reproduction of the 
originally captured sounds. 

It is another object of the present invention to provide a method and 
apparatus for reproducing sounds produced by a plurality of sound sources, where 
sounds from each sound source (or a predetermined group of sources) are captured 
by separate sound detectors, and where the separately captured sounds are 
converted to audio signals, each of which is transmitted separately to a separate 
loudspeaker system for reproduction of the originally captured sounds. 

It is another object of the present invention to provide a method and 
apparatus for reproducing a plurality of separately recorded sounds or sounds 
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produced by a plurality of sound sources, where sounds from each source (or a 
predetermined group of sources) are captured by separate sound detectors, and 
where the separately captured sounds are converted to audio signals, each of 
which is transmitted separately to a separate loudspeaker system for reproduction 
of the originally captured sounds (with or without first recording the audio 
signals), where each loudspeaker system comprises a plurality of loudspeakers or 
a plurality of groups of loudspeakers (e.g., loudspeaker clusters) customized for 
reproduction of specific types of sound sources or group(s) of sound sources. 
Preferably the customization is based at least in part on characteristics of the 
sounds to be reproduced by the loudspeaker or based on the dynamic behavior of 
the sounds or groups of sounds. 

It is another object of the present invention to provide a method and 
apparatus for reproducing a plurality of separately recorded sounds or sounds 
produced by a plurality of sound sources, where sounds from each sound source 
(or a predetermined group of sources) are captured by separate sound detectors, 
and where the separately captured sounds are converted to audio signals, each of 
which is transmitted separately to a separate loudspeaker system for reproduction 
of the originally captured sounds (with or without first recording the audio 
signals), where each signal path is connected to a separate amplification systems 
to separately amplify audio signals corresponding to the sounds from each source 
(or predetermined group of sources). The amplifier systems may be customized 
for the particular characteristics of the audio signals that it will be amplifying. 

It is another object of the present invention to provide a method and 
apparatus for reproducing a plurality of separately recorded sounds or sounds 
produced by a plurality of sound sources, where sounds from each sound source 
(or a predetermined group of sound sources) are captured by separate sound 
detectors, and where the separately captured sounds are converted to audio signals, 
each of which is transmitted to a separate loudspeaker system for reproduction of 
the originally captured sounds (with or without first recording the audio signals), 
where each signal path is connected to an amplification system to separately 
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amplify audio signals corresponding to the sounds from each source (or 
predetermined group of sources) and where the amplifier systems are separately 
controlled by a controller so that the relationship among the components of the 
power (amplifier) network and those of the loudspeaker network can be selectively 
controlled. This control can be automatically implemented based on the dynamic 
characteristics of the audio signals (or the produced sounds) or a user can 
manually control the reproduction of each sound (or predetermined groups of 
sounds) through a user interface that enables the user to independently adjust the 
input power levels of each sound (or predetermined group of sounds) from "off" 
to relatively high levels of corresponding output power levels without necessarily 
affecting the power level of any of the other independently controlled audio 
signals. 

SUMMARY OF THE INVENTION 

To accomplish these and other objects of the present invention, 
improved methods and apparatus for recording and/or reproducing sound are 
disclosed. According to one embodiment, a method and apparatus for recording 
and reproducing sound comprises a plurality of sound sources or predetermined 
groups of sound sources for concurrently producing sounds, a plurality of 
detectors for detecting sound waves from respective ones of the sound sources or 
from respective ones of the groups of sound sources and converting each of the 
detected sound waves to separate audio signals without mixing the audio signals 
and separately transmitting each of the audio signals to one of a plurality of 
loudspeaker systems for reproduction. 

If desired, the audio signals output from the sound detectors may be 
recorded on a recording medium for subsequent readout prior to being transmitted 
to the loudspeaker systems for reproduction. If recorded, preferably the recording 
mechanism separately records each of the audio signals on the recording medium 
without mixing the audio signals. Subsequently, the stored audio signals are 
separately retrieved and are provided over separate signal paths to individual 
amplifier systems and then to the separate loudspeaker systems. Preferably, the 
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audio signals are separately controllable, either automatically or manually. The 
amplifier and loudspeaker systems for each signal path may be automatically 
controlled by a dynamic controller that controls the relationship among the 
amplifier systems, the components of the amplifier systems, the loudspeaker 
systems and the components of the of the loudspeaker systems. For example, the 
controller can individually turn on/off individual amplifiers of an amplifier system 
so that increased/decreased power levels can be achieved by using more or less 
amplifiers for each audio signal instead of stretching the range of a single 
amplifier. Similarly, the controller can control individual loudspeakers within a 
loudspeaker system. 

The loudspeaker systems preferably are each made up of one or 
more loudspeakers or loudspeaker clusters and are customized for reproduction of 
specific types of sounds produced by the respective sound source or group of 
sound sources associated with the signal path. For example, a loudspeaker system 
may be customized for the reproduction of violins or stringed instruments. The 
customization may take into account various characteristics of the sounds to be 
reproduced, including, frequency, directivity, etc. Additionally, the loudspeakers 
for each signal path may be configured in a loudspeaker cluster that uses an 
explosion technique, i.e., sound radiating from a source outwards in various 
directions (as naturally produced sound does) rather than using an implosion 
technique, i.e., sound projecting inwardly toward a listener (e.g., from a perimeter 
of speakers as with surround sound or from a left/right direction as with stereo). 
In other circumstance, an implosion technique or a combination of 
explosion/implosion may be preferred. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic illustration of a sound capture and recording 
system according to one embodiment of the present invention. 

Fig. 2 is a schematic illustration of a sound reproduction system 
according to one embodiment of the present invention. 
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Fig. 3 is a schematic illustration of an exploded view of an amplifier 
system and loudspeaker system for one signal path according to one embodiment 
of the present invention. 

Fig. 4 is a schematic illustration of an example configuration for an 
annunciator according to one embodiment of the present invention. 

Fig. 5 is a schematic illustration of an example configuration for an 
annunciator according to one embodiment of the present invention. 

Fig. 6 is a schematic illustration of an example configuration for an 
annunciator according to one embodiment of the present invention. 
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 1 is a schematic illustration of a sound capture and recording 
system according to one embodiment of the present invention. As shown in Fig. 
1, the system comprises a plurality of sound sources (SS r SS N ) for producing a 
plurality of sounds, a plurality of sound detectors (SD r SD N ), such as 
microphones, for capturing or detecting the sounds produced by the N sound 
sources and for separately converting the N sounds to N separate audio signals. 
As shown in Fig. 1, the N separate audio signals may be conveyed over separate 
signal paths (SP r SP N ) to be recorded on a recording medium 40. Alternatively, 
the N separate audio signals may be transmitted to a sound reproduction system 
(such as shown in Fig. 2), which preferably includes N loudspeaker systems for 
converting the audio signals to sound. If the audio signals are to be recorded, the 
recording medium 40 may be, e.g., an optical disk on which digital signals are 
recorded. Other storage media (e.g., tapes) and formats (e.g., analog) may be 
used. In the event that digital recording is used, the N audio signals are separately 
provided over N signal paths to an encoder 30. Any suitable encoder can be used. 
The outputs of the encoder 30 are applied to the recording medium 40, where the 
signals are separately recorded on the recording medium 40. Multiplexing 
techniques (e.g., time division multiplexing) may also be used. If no recording is 
performed, the output of the acoustical manifold 10 or the sound detectors (SD r 
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SD N ) may be supplied directly to the amplifier network 70 or acoustical manifold 
60 (Fig. 2). 

If desired, the N audio signals output from the N sound detectors 
(SD r SD N ) may be input to an acoustical manifold 10 and/or an annunciator 20 
prior to being input to encoder 30. The acoustical manifold 10 is an input/output 
device that receives audio signal inputs, indexes them (e.g., by assigning an 
identifier to each data stream) and determines which of the inputs to the manifold 
have a data stream (e.g., audio signals) present. The manifold then serves as a 
switching mechanism for distributing the data streams to a particular signal path 
as desired (detailed below). The annunciator 20 can be used to enable flexibility 
in handling different numbers of audio signals and signal paths. Annunciators are 
active interface modules for transferring or combining the discrete data streams 
(e.g., audio signals) conveyed over the plurality of signal paths at various points 
within the system from sound capture to sound reproduction. For example, when 
the number of signal paths output from the sound detectors is equal to the number 
of amplifier systems and/or loudspeaker systems, the function of the annunciator 
can be passive (no combining of signals is necessarily performed). When the 
number of outputs from the sound detectors is greater than the number of amplifier 
systems and/or loudspeaker systems, the annunciator can combine selected signal 
paths based on predetermined criteria, either automatically or under manual 
control by a user. For example, if there are N sound sources and N sound 
detectors, but only N-l inputs to the encoder are desired, a user may elect to 
combine two signal paths in a manner described below. The operation and 
advantages of these components are further detailed below. 

Fig. 2 schematically depicts a sound reproduction system according 
to a preferred embodiment of the invention. It can be used with the sound 
capture/recording system of Fig. 1 or with other systems. This portion of the 
system may be used to read and reproduce stored audio signals or may be used to 
receive audio signals that are not stored (e.g., a live feed from the sound detectors 
SD r SD N ). When it is desired to reproduce sounds based on the stored audio 
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signals, the stored audio signals are read by a reader/decoder 50. The reader 
portion may include any suitable device (e.g., an optical reader) for retrieving the 
stored audio signals from the storage medium 40 and, if necessary or desired, any 
suitable decoder may be used. Preferably, such a decoder will be compatible with 
the encoder 30. The separate audio signals from the reader/decoder 50 are 
supplied over signal paths to an amplifier network 70 and then to a loudspeaker 
network 80 as detailed below. Prior to being supplied to the amplifier network 70, 
the audio signals from reader/decoder 50 may be supplied to annunciator 60. 

For simplicity, it will be assumed that N audio signals are input to 
annunciator 60 and that N audio signals are output therefrom. It is to be 
understood, however, that different numbers of signals can be input to and output 
from annunciator 20. If, for example, only five audio signals are output from 
annunciator 60, only five amplifier systems and five loudspeaker systems are 
necessary. Additionally, the number of audio signals output from annunciator 60 
may be dictated by the number of amplifier or loudspeaker systems available. For 
example, if a system only has four amplifier systems and four loudspeaker 
systems, it may be desirable for the annunciator to output only four audio signals. 
For example, the user may elect to build a system modularly (i.e., adding amplifier 
systems and loudspeaker systems one or more at a time to build up to N such 
systems). In this event, the annunciator facilitates this modularity. The user 
interface 55 enables the user to select which audio signals should be combined, if 
they are to be combined, and to control other aspects of the systems as detailed 
below. 

Referring to Figs. 2 and 3, the amplifier network 70 preferably 
comprises a plurality of amplifier systems AS r AS N each of which separately 
amplifies the audio signals on one of the N signal paths. As shown in Fig. 3, each 
amplifier system may comprise one or more amplifiers (A-N) for separately 
amplifying the audio signals on one of the N signal paths. From the amplifier 
network 70, each of the audio signals are supplied over separate signal paths to a 
loudspeaker network 80. The loudspeaker network 80 comprises N loudspeaker 
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systems LS r LS N each of which separately reproduces the audio signals on one of 
the N signal paths. As shown in Fig. 3, each loudspeaker system preferably 
includes one or more loudspeakers or loudspeaker clusters (A-N) for separately 
reproducing the audio signals on each of the N signal paths. 

Preferably, each loudspeaker or loudspeaker cluster is customized 
for the specific types of sounds produced by the sound source or groups of sound 
sources associated with its signal path. Preferably, each of the amplifier systems 
and loudspeaker systems are separately controllable so that the audio signals sent 
over each signal path can be controlled individually by the user or automatically 
by the system as detailed below. More preferably, each of the individual 
amplifiers (A-N) and each of the individual loudspeakers (A-N) are each 
separately controllable. For example, it is preferable that each of amplifiers A-N 
for amplifier system AS! is separately controllable to be on or off, and if on to 
have variable levels of amplification from low to high. In this way, power levels 
of audio signals on that signal path may be stepped up or down by turning on 
specific amplifiers within an amplifier system and varying the amplification level 
of one or more of the amplifiers that are on. Preferably, each of the amplifiers of 
an amplifier system is customized to amplify the audio signals to be transmitted 
through that amplifier system. For example, if the amplifier system is connected 
in a signal path that is to receive audio signals corresponding to sounds that consist 
of primarily low frequencies (e.g., bass sounds from a drum), each of the 
amplifiers of that amplifier system may be designed to optimally amplify low 
frequency audio signals. This is an advantage over using amplifiers that are 
generic to a broad range of frequencies. Moreover, by providing multiple 
amplifiers within one amplifier system for a specific type of audio signal (e.g., 
sounds that consist of primarily low frequencies), the power level output from the 
amplifier system can be stepped up or down by Uiniing on or off individual 
amplifiers. This is an advantage over using a single amplifier that must be varied 
from very low power levels to very high power levels. Similar advantages are 
achieved by using multiple loudspeakers within each loudspeaker system. For 
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example, two or more loudspeakers operating at or near a middle portion of a 
power range will reproduce sounds with less distortion than a single loudspeaker 
at an upper portion of its power range. Additionally, loudspeaker arrays may be 
used to effect directivity control over 360 degrees or variations thereof. 

As also shown in Fig. 2, the present invention may include a user 
interface 55 to provide a user with the ability to manually manipulate the audio 
signals on each signal path independently of the audio signals on each of the other 
signal paths. This ability to manipulate includes, but is not limited to, the ability 
to manipulate: 1) master volume control (e.g., to control the volume or power on 
all signal paths); 2) independent volume control (e.g., to independently control the 
volume or power on one or more individual signal paths); 3) independent on/off 
power control (e.g., to turn on/off individual signal paths); 4) independent 
frequency control (e.g., to independently control the frequency or tone of 
individual signal paths); 5) independent directional and/or sector control (e.g., to 
independently control sectors within individual signal paths and/or control over 
the annunciator. 

Preferably, the user interface 55 includes a master volume control 
(MC) and N separate controls (C r C^ for the N signal paths. A dynamics 
override control (DO) may also be provided to enable a user to manually override 
the automatic dynamic control of dynamic controller 90. 

Also shown in Fig. 2 is a dynamic control module 90, which can 
provide separate control of the amplifier systems (AS^ASn), the loudspeaker 
systems (LS r LS N ) and the annunciators 20, 60. Dynamics control module 90 is 
preferably connected to the user interface 55 (e.g., directly or via annunciator 60) 
to permit user interaction and manual control of these components. 

According to one aspect of the invention, dynamics control module 
90 includes a controller 91, one or more annunciator interfaces 92, one or more 
amplifier system interfaces 93, one or more loudspeaker interfaces 94 and a 
feedback control interface 95. The annunciator interface 92 is connected to one 
or more annunciators (20, 60). The amplifier interface 93 is operatively connected 
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to the amplifier network 70. The loudspeaker interface 94 is connected to the 
loudspeaker network 80. Dynamics control module 90 controls the relationship 
among the amplifier systems and loudspeaker systems and the individual 
components therein. Dynamics control module 90 may receive feedback via the 
feedback control interface 95 from the amplification network 70 and/or the 
loudspeaker network 80. Dynamics control module 90 processes signals from 
amplification network 70 and/or sounds from loudspeaker network 80 to control 
amplification network 70 and loudspeaker network 80 and the components thereof. 
Dynamics control module 90 preferably controls the power relationship among the 
amplifier systems of the amplification network 70. For example, as power or 
volume of an amplifier system is increased, the dynamic response of a particular 
audio signal amplified by that amplifier system may vary according to 
characteristics of that audio signal. Moreover, as the overall power of the 
amplifier network is increased or decreased, the dynamic relationship among the 
audio signals in the separate signal paths may change. Dynamics control module 
90 can be used to discretely adjust the power levels of each amplifier system based 
on predetermined criteria. An example of the criteria on which dynamics control 
module 90 may base its adjustment is the individual sound signal power curves 
(e.g., optimum amplification of audio signals when ramping power up or down 
according to the power curves of the original sound event). Module 90 can 
discretely activate, deactivate, or change the power level of, any of the 
amplification systems 70 AS r AS N and preferably, the individual components 
(A-N) of any given amplifier system AS r AS N . 

Module 90 can also control the loudspeaker network 80 based on 
predetermined criteria. Preferably, module 90 can discretely activate, deactivate, 
or adjust the performance level of each individual loudspeaker system and/or the 
individual loudspeakers or loudspeaker clusters (A-N) within a loudspeaker system 
(LS!-LS N ). Thus, the system components are capable of being individually 
manipulated to optimize or customize the amplification and reproduction of the 
audio signals in response to dynamic or changing external criteria (e.g., power), 
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sound source characteristics (e.g., frequency bandwidth for a given source), and 
internal characteristics (e.g., the relationship between the audio signals of the 
different signal paths). 

The user interface 55 and/or dynamic controller 90 enables any 
signal path or component to be turned on/off or to have its power level controlled 
either automatically or manually. The dynamic controller 90 also enables 
individual amplifiers or loudspeakers within an amplifier system or loudspeaker 
system to be selectively turned on depending, for example, on the dynamics of the 
signals. For example, it is advantageous to be able to turn on two amplifiers 
within one system to increase the power level of a signal rather than maxing out 
the amplification of a single amplifier which can cause undesired distortion. 

As will be apparent from the foregoing description, whether the N 
separate audio signals are recorded first and then reproduced or reproduced 
without first being recorded, the present invention enables various types of control 
to be effected to enable the reproduced sounds to have desired characteristics. 
According to one embodiment, the N separate audio signals output from the sound 
detectors (SD r SD N ) are maintained as N separate audio signals throughout the 
system and are provided as N separate inputs to the N loudspeaker systems. 
Typically, it is desired to do this to accurately reproduce the originally captured 
sounds and avoid problems associated with mixing of audio signals and/or sounds. 
However, as detailed herein various types of selective control over the audio 
signals can be effected by using acoustical manifold 10, one or more annunciators 
(20, 60), a user interface 55 and a dynamic controller 90 to enable various types 
of desired mixing of audio signals to permit modular expansion of a system. For 
example, one or more acoustical manifolds 10 can be used at various points in the 
system to enable audio signals on one signal path to be switched to another signal 
path. For example, if the sounds produced by SSI are captured by SD1 and 
converted to audio signals on signal path SP1, it may be desired to ultimately 
provide these audio signals to loudspeaker system LS 4 (e.g., since the loudspeakers 
may be customized for a particular type of sound source). If so, then the audio 
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signals input to the acoustical manifold 10 on SP1 are routed to output 4 of the 
acoustical manifold 10. Other signals may be similarly switched to other signal 
paths at various points within the system. Thus, if the characteristics of the sounds 
produced by a sound source (SS) as captured by a sound detector (SD) change, the 
acoustical manifold 10 enables those signals to be routed to an amplifier system 
and/or loudspeaker system that is customized for those characteristics, without 
reconfiguring the entire system. 

One or more annunciators (e.g., 20, 60) may be used to selectively 
combine two or more audio signals from separate signal paths or it can permit the 
N separate audio signals to pass through all or portions of the system without any 
mixing of the audio signals. One advantage of this is where there are more sound 
detectors then there are amplifier systems or loudspeaker systems. Another is 
when there are less amplifier systems and/or loudspeaker systems than there are 
signal paths. In either case (or in other cases) it may be desired to selectively 
combine audio signals corresponding to the sounds produced by two or more 
sound sources. Preferably, if such sounds or audio signals are mixed, selective 
mixing is performed so that signals having common characteristics (e.g., 
frequency, directivity, etc.) are mixed. This also enables modular expansion of the 
system. 

As will be apparent from the foregoing, during the entire process 
from the detection of the sound to its reproduction by the loudspeakers, each of 
the audio signals corresponding to sounds produced by a sound source are 
preferably maintained separate from other sounds/audio signals produced by 
another sound source. Unless specifically desired to do so, the signals are not 
mixed. In this way, many of the problems with prior systems are avoided. While 
the foregoing discussion addresses the use of separate signal paths to keep the 
audio signals separate, it is to be understood that this may also be accomplished 
by multiplexing one or more signals over a signal path while maintaining the 
information separate (e.g., using time division multiplexing). 
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If desired, a feedback system 100 (Fig. 2) may be provided. If used, 
it can serve at least two primary functions. The first relates to acoustical data 
acquisition and active feedback transmission. This is accomplished, for example, 
by use of diagnostic transducers DT r DT N that measure the output data (e.g., 
sounds) exiting each port of the system (e.g., each loudspeaker system), providing 
feedback to the dynamics control module 90 via the feedback control interface 95. 
The dynamics control module 90 then controls the system components according 
to a predetermined control scheme. A second function relates to the dynamic 
control schemes. The dynamics control module 90 controls the macro/micro 
relationships between playback system components, systems, and subsystems 
under dynamic conditions. The dynamics module 90 controls the micro 
relationships among the components (e.g., amplifiers and/or loudspeakers within 
a single signal path) and the macro relationships among the separate signal paths. 
The micro relationships include the relationship between individual amplifiers 
within a given amplifier system (e.g., where each signal path has its own discrete 
amplifier system with one or more amplifiers) and/or the micro relationships 
between individual loudspeakers within a given loudspeaker system (e.g., where 
each signal path has its own discrete loudspeaker system with one or more 
loudspeakers). The macro relationships include the relationships among the 
amplifier systems and loudspeaker systems of the separate signal paths. Such 
control is implemented according to predetermined criteria or control schemes 
(e.g., based on the characteristics the original sound, the acoustics of the venue, 
the desired directivity patterns, etc.). Such control schemes can be embedded in 
the audio signals of each signal path, permanently hard-coded into the amplifier 
system for each signal path, or determined by active feedback signals originating 
from feedback system 100 based on the actual sounds produced. The dynamics 
control module 90 can control the macro relationships between the discrete 
presentation channels as the dynamics of the systems change (e.g., changes in 
master volume control, changes in the playback system configuration, changes in 
the venue dynamics, changes in recording methods/accuracies, changes in music 
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type, etc.). Diagnostic channels can include a number of active and passive 
feedback paths linking the output data from each signal path to a control module 
which, in turn, communicates a predetermined control scheme to each signal path 
and/or specific discrete signal paths. A purpose of the diagnostic system is to 
provide a method for controlling the interaction between individual sounds within 
a given sound field as the dynamics of each sound change in proportion to changes 
in volume levels and/or changes in the dynamics of the performance venue. 

By way of example, Figs. 4, 5 and 6 depict various configurations 
for a system having multiple stages (ST r ST 3 ) and multiple annunciators (AN r 
ANj). Fig. 4 depicts N signals input but only five outputs. Fig. 5 depicts N inputs 
with four outputs. Fig. 6 depicts N inputs and only two outputs. In each of Figs. 
4-6, the various stages can be Capture, Transmission (e.g., recording or live feed) 
and Presentation stages. Other stages can be used. For example, the Capture stage 
may include a first number of signal paths to capture the sounds produced by the 
sound sources. Preferably, there is one signal path for each sound source, but 
more or less may be used. The Transmission stage may include a second number 
of signal paths between the Capture stage and the recording medium and/or other 
portions (e.g., playback) of the system or transmitted to a "live feed" network. 
The second number of signal paths may be greater than, less than or equal to the 
first number of signal paths. The Presentation stage may include a third number 
of signal paths for reproduction of the sounds so that separate amplifier and 
loudspeaker systems may be used for each signal path. The third number of signal 
paths may be greater than, less than or equal to the first and or second number of 
signal paths. Preferably, the first, second and third number of signal paths are 
equal to enable independence throughout the Capture, Transmission and 
Presentation stages. When the number of signal paths are not equal, however, the 
annunciator module serves to control the signal paths and routing of signals 
thereover. 

For purposes of example only, the sound sources SS r SS N may 
include keyboards (e.g., a piano), strings (e.g., a guitar), bass (e.g., a cello), 
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percussion (e.g., a drum), woodwinds (e.g., a clarinet), brass (e.g., a saxophone), 
and vocals (e.g., a human voice). These seven identified sound sources represent 
the seven major groups of musical sound sources. The invention does not require 
seven sound sources. More or less can be used. Of course, other sound sources 
or groups of sound sources may be also be used as indicated by box SS N . In the 
general case, N sound sources may be used where N is an integer greater than 1, 
or equal, but preferably greater than 1. It is well known that each of these seven 
major groups of musical sound sources have different audio characteristics and 
that, while each individual sound source within a group may have significant tonal 
differences (i.e., the violin and guitar), the sound sources within a group may have 
one or more common characteristics. 

According to one aspect of the present invention, the sounds 
produced by each of the N sound sources SS r SS N are separately detected by one 
of a plurality of sound detectors SD r SD N , for example, N microphones or 
microphone sets. Preferably, the sound detectors are directional to detect sound 
from substantially only one or selected ones of the plurality of sound sources. 
Each of the N sound detectors preferably detect sounds produced by one of the N 
sound sources and converts the detected sounds to audio signals. If each of the N 
sound sources simultaneously produces sound, then N separate audio signals will 
exist. Each sound detector may comprise one or more sound detection devices. 
For example, each sound detector may comprise more than one microphone. 
According to a preferred embodiment, three microphones (left, right and center) 
are used for each sound source. As detailed below, the use of these microphones 
is just one example of the use of a plurality of sound detection devices for each 
sound source. In other situations, more or less may be desired. For example, it 
may be desirable to surround a source with a plurality of microphones to obtain 
more directional information. The audio signals output from each of the N sound 
detectors or sound detection devices are supplied over a separate signal path as 
described above. 
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Each signal path may comprise multiple channels. For example, as 
shown in Fig. 1, each signal path may include a plurality of channels, (e.g., a left, 
right and center channel). In the general case, each signal path comprises M 
channels, where M is an integer greater than or equal to 1. However, it is not 
necessary for each signal path to have the same number of channels. For 
simplicity of discussion, it will be assumed that there are M channels for each of 
the N signal paths. 

The number of channels for a particular signal path need not be 
limited to three. More or fewer channels may be incorporated as desired. For 
example, a plurality of channels may be used to provide directional control (e.g., 
left, right and center). However, some or all of the channels may be used to 
provide frequency separation or for other purposes. For example, if three channels 
are used, each of the three channels could represent one musical instrument within 
a given group. For example, the musical group may be "strings" (e.g., if the event 
being recorded has two violins and one acoustical guitar). In this case, one 
channel could be used for one violin, another channel could be used for the second 
violin, and the third channel could be used for the acoustical guitar. Another use 
of separate channels is to enable power stepping, where one channel is used for 
audio signals up to a first level, then a second channel is added as the power level 
is increased above the first level, and so on. This method helps regulate the 
optimum efficiency level for each of the loudspeakers used in the loudspeaker 
network. 

The recording process, if used, generally involves separately 
recording the M X N audio signals onto the recording medium 40 to enable the 
M X N signals to be subsequently read out and reproduced separately. The 
recording and read out may be accomplished in a standard manner by providing 
independent recording/reading heads for each signal path/channel or by time- 
division multiplexing the audio signals through one or more recording/reading 
heads onto or from M X N tracks of the recording medium. 
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According to another aspect of the invention, the separately recorded 
audio signals are separately reproduced. As shown in Fig. 2, the reproduction of 
the audio signals includes separately retrieving the M X N signals by playback 
mechanism 50 (and performing any necessary or desired decoding). Then the 
audio signals are supplied over N separate signal paths (where each signal path 
may have M channels) to an amplifier network 70 having N amplifier systems and 
providing the output of the N amplifier systems to loudspeaker network 80, which 
preferably comprises N loudspeaker systems. Each loudspeaker system may 
comprise M X N loudspeakers or a greater or lesser number of loudspeakers, as 
detailed below. 

According to one embodiment of the present invention, each sound 
source may be a group of sound sources instead of an individual source. 
Preferably, each group includes sound sources with one or more similar 
characteristics. For example, these characteristics may include musical groupings 
(keyboards, strings, bass, percussion, woodwinds, brass group, and vocals), 
frequency bandwidth, or other characteristics. Thus, if more than one type of 
string instruments is used, it may be acceptable to use one signal path for the string 
instruments and separate signal paths, etc. for other sound sources or groups of 
sound sources. This still enables recognition of the advantages derived from the 
use of customized loudspeaker systems since sounds with common characteristics 
are produced by the same loudspeaker system. 

According to one embodiment, the criteria used for grouping sound 
sources is related to a common dynamic behavior of particular audio signals when 
they are amplified. For example, a particular amplifier may have different 
distortion effects on different audio signals having different characteristics (e.g., 
frequency bandwidth). Thus, it also may be preferable to use a different type of 
amplifier system for different types of audio signals. Another criteria used for 
grouping sound sources is common directivity patterns. For instance, "horns" are 
very directional and can be grouped together while "keyboard instruments" are 
less directional than horns and would not be compatible with the "horns" 
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customized speaker configuration, and therefore would not be grouped together 
with horns. 

The sound system need not be limited to any particular number of 
signal paths. The number of signal paths can be increased or decreased to 
accommodate larger or smaller numbers of individual sound sources or sound 
groups. Further, application of the system is not limited to musical instruments 
and vocals. The sound system has many applications including standard movie 
theater sound systems, special movie theaters (e.g., OmniMax, IMAX, Expos) 
cyberspace/computer music, home entertainment, automobile and boat sound 
systems, modular concert systems (e.g., live concerts, virtual concerts), auto 
system electronic crossover interface, home system electronic crossover interface, 
church systems, audio/visual systems (e.g., advertising billboards, trade shows), 
educational applications, musical compositions, and HDTV applications, to name 
but a few. 

Preferably, loudspeaker network 80 consists of several loudspeaker 
systems, each including a plurality of loudspeakers or loudspeaker clusters each 
of which is used for one of the signal paths. Each loudspeaker cluster includes 
one or more loudspeakers customized for the type of sounds that it is used to 
reproduce. A given loudspeaker cluster may be responsive to the power change 
of the corresponding amplification system. For example, if the power level 
supplied to a given loudspeaker network is below a first predetermined level, one 
or a group of loudspeaker components may be active to reproduce sound. If the 
power level exceeds the first predetermined level, a second or second group of 
loudspeaker components may become active to reproduce the sound. This avoids 
overloading the first loudspeaker (or first group of loudspeakers) and also avoids 
under powering the loudspeaker s(s). Thus, depending on the power level of the 
audio signals on one (or more) of the signal paths, the individual loudspeakers 
within a given loudspeaker cluster can be automatically activated or deactivated 
(e.g., manually or automatically under control of the dynamics control module 90). 
Furthermore, a control signal embedded in the audio signal can identify the type 
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of sound being delivered and thus trigger the precise group(s) of speakers, within 
a loudspeaker cluster, that most closely represents the characteristics of that signal 
(e.g., actual directivity pattern(s) of the sound source(s) being reproduced). For 
example, if the sound source being reproduced is a trumpet, the embedded control 
signal would trigger a very narrow group of speakers within the larger loudspeaker 
network, since the directivity of an actual trumpet is relatively narrow. Similar 
control can occur for other characteristics. 

The audio signals, if digital, preferably are encoded and decoded at 
a sample rate of at least 88.2 KHz and 20-bit linear quantization. Other sample 
rates and quantization rates can be used however. 

The foregoing is not intended to limit the scope of the invention. 
The invention is only limited by the claims appended hereto. 
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